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Abstract 


The  Advanced  Integrated  Multi-sensing  Surveillance  (AIMS)  system  technology  demonstration 
program  (TDP)  was  designed  to  improve  the  surveillance  and  reconnaissance  capabilities  of  the 
Canadian  Forces  (CF),  particularly  at  night  and  in  degraded  weather  conditions.  Thereby,  to 
evaluate  the  utility  of  this  system  for  Search  and  Rescue  (SAR)  operations,  evaluation  criteria 
were  developed  and  presented  in  previous  publications.  Also,  to  improve  the  previous  published 
database,  this  memorandum  presents  the  analysis  of  the  data  collected  at  SAREX  2007  using  the 
same  evaluation  criteria. 


The  first  part  of  this  memorandum  presents  a  short  description  of  the  SAREX  2007  search  event. 
Then,  SAREX  2007  results  are  presented  and  the  analysis  of  the  data  is  made  using  different 
evaluation  criteria  such  as:  target  location  accuracy,  target  identification,  number  of  call-arounds, 
false  detection  rate,  probability  of  detection  and  number  of  targets.  Finally,  a  discussion  is 
presented  and  recommendations  are  provided. 


Resume 


Le  programme  de  demonstration  de  technologies  (PDT)  du  systeme  perfectionne  de  surveillance 
multi-capteurs  integre  (AIMS)  a  ete  congu  pour  ameliorer  les  capacites  de  surveillance  et  de 
reconnaissance  des  Forces  canadiennes  (FC),  particulierement  la  nuit  et  dans  des  conditions 
meteorologiques  degradees.  Ainsi,  pour  evaluer  Tutilite  du  systeme  lors  des  operations  de 
recherche  et  sauvetage  (SAR),  des  criteres  d’ evaluation  ont  ete  developpes  et  presentes  dans  des 
publications  precedentes.  Aussi,  pour  augmenter  l’etendue  des  donnees  publiees  precedemment, 
ce  memorandum  presente  1’ analyse  des  donnees  recueillies  lors  du  SAREX  2007  en  utilisant  les 
memes  criteres  d’ evaluation. 

La  premiere  partie  du  memorandum  presente  une  courte  description  de  l’evenement  de  recherche 
du  SAREX  2007.  Ensuite,  les  donnees  du  SAREX  2007  sont  presentees  et  Tanalyse  des  donnees 
est  effectuee  a  l’aide  de  criteres  devaluation  tels  que  :  precision  de  la  position  d’une  cible, 
identification  d’une  cible,  nombre  de  retours  en  arriere,  taux  de  fausses  detections,  probability  de 
detection  et  nombre  de  cibles.  Finalement,  une  discussion  et  des  recommandations  sont 
formulees. 
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Executive  summary 


SAREX  2007  search  event  data  analysis 

G.  Toussaint;  DRDC  Valcartier  TM  2008-329;  Defence  R&D  Canada  -  Valcartier; 
February  2009. 

The  Advanced  Integrated  Multi-sensing  Surveillance  (AIMS)  system  technology  demonstration 
program  (TDP)  was  designed  to  improve  the  surveillance  and  reconnaissance  capabilities  of  the 
Canadian  Forces  (CF),  particularly  at  night  and  in  degraded  weather  conditions.  Since  the  intent 
is  that  AIMS  participate  in  the  SAREX  2009,  it  was  considered  useful  for  the  operational  research 
(OR)  analyst  attached  to  the  project  to  attend  previous  SAREX  exercises  to  collect  data  from 
experienced  crews  flying  on  different  platforms.  The  analysis  of  the  data  collected  will  help 
evaluate  the  utility  for  the  CF  to  use  the  AIMS  system  during  search  and  rescue  (SAR) 
operations.  Thus,  by  using  some  of  the  evaluation  criteria  developed  in  previous  publications,  this 
memorandum  presents  the  analysis  of  the  data  collected  at  SAREX  2007,  which  improves  the 
database  presented  in  these  publications. 

The  main  conclusions  drawn  from  the  data  analysis  of  this  memorandum  are: 

1.  Target  location  accuracy:  As  expected,  few  targets  were  detected  under  the  aircraft  since 
part  of  this  area  is  out  of  reach  for  SAR  technicians  (techs)/spotters.  Also,  all  the  targets  were 
detected  within  0.8  nautical  miles  (NM),  which  is  within  the  1  NM  expected. 

2.  Target  identification:  It  was  not  possible  to  demonstrate  which  one,  between  colour  and 
shape,  was  the  most  useful  parameter  to  detect  an  object  from  its  background.  Therefore,  it 
could  be  suitable  for  AIMS  to  be  equipped  with  a  high  definition  colour  screen  to  maximize 
the  chances  of  detecting  a  target. 

3.  Number  of  call-arounds:  The  majority  of  call-arounds  were  made  for  nothing.  It  would  be 
preferable  to  evaluate  this  criterion  during  a  search  when  there  would  be  sufficient  time  to 
perform  a  good  search. 

4.  False  detection  rate:  It  was  not  possible  to  draw  any  conclusion  from  the  false  detection  rate. 
It  is  suggested  to  compare  only  the  false  detection  rate  obtained  using  AIMS  to  that  obtained 
without  AIMS  for  the  same  SAREX  event. 

5.  Probability  of  detection:  It  was  not  sufficient  to  use  only  the  data  obtained  from  SAREX 
2007  to  reach  a  conclusion  on  the  probability  of  detection  (POD).  However,  combining  the 
data  from  1988  to  1991,  1993  and  2004  to  2007,  we  can  confirm  that  the  POD  for  passive 
targets  is  significantly  less  than  that  for  cooperative  targets. 

6.  Number  of  targets:  Past  search  events  data  were  compiled  to  determine  the  number  of 
targets  that  were  necessary  to  demonstrate,  with  95%  confidence,  an  improvement  in  the 
POD.  Since  we  do  not  know  yet  the  POD  of  the  SAREX  search  event  in  which  AIMS  will 
participate,  these  results  can  only  be  used  as  an  indication  of  the  number  of  targets  required. 
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Nevertheless,  as  many  targets  as  possible  should  be  employed  for  the  SAREX  search  event  in 
which  AIMS  will  participate. 

The  analyst  on  the  AIMS  TD  (technology  demonstration)  should  gather  data  at  SAREX  2008  and 
should  be  ready  for  the  AIMS  flight  trials  in  SAREX  2009.  Also,  as  discussed  in  a  previous 
memorandum,  it  is  important  to  mention,  that  if  the  use  of  AIMS  at  the  search  event  shows  an 
equal  or  a  better  performance  than  not  using  it,  then  it  will  be  possible  to  state  that  the  use  of  this 
system  will  be  very  useful  for  SAR  operations.  However,  the  opposite  would  not  necessarily  be 
true  since  some  factors  present  in  a  SAREX  search  event  would  be  absent  in  a  real  SAR  incident 
and  could  influence  negatively  the  performance  of  the  AIMS  system.  Some  of  these  factors  are: 
AIMS  will  be  flying  with  a  different  type  of  aircraft  than  the  other  teams  and  they  will  have  fewer 
SAR  techs/spotters  than  what  is  normally  prescribed  for  a  SAREX  search  event.  Also,  after  the 
AIMS  trials  at  the  SAREX,  only  one  set  of  data  will  be  used  for  comparison  purposes  and  no 
search  will  take  place  at  night  time  and  in  bad  weather,  conditions  for  which  the  system  was 
especially  designed  to  improve  search  efficiency.  Finally,  in  a  SAREX  search  event,  there  are 
usually  more  cooperative  targets  than  passive  targets  whereas  the  AIMS  system  has  sensors  that 
could  help  finding  passive  targets. 


IV 
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SAREX  2007  search  event  data  analysis 

G.  Toussaint;  DRDC  Valcartier  TM  2008-329;  R  &  D  pour  la  defense  Canada  - 
Valcartier;  Fevrier  2009. 

Le  programme  de  demonstration  de  technologies  (PDT)  du  systeme  perfectionne  de  surveillance 
multi-capteurs  integre  (AIMS)  a  ete  congu  pour  ameliorer  les  capacites  de  surveillance  et  de 
reconnaissance  des  Forces  canadiennes  (FC),  particulierement  la  nuit  et  dans  des  conditions 
meteorologiques  degradees.  Etant  donne  que  l’intention  est  qu’AIMS  participe  au  SAREX  2009, 
on  a  juge  utile  que  l’analyste  en  recherche  operationnelle  (OR)  affecte  au  projet  assiste  aux 
exercices  precedents  du  SAREX  pour  recueillir  des  donnees  d’ equipages  experiments  volant  sur 
differentes  plates-formes.  L’analyse  des  donnees  recoltees  va  contribuer  a  evaluer  1’utilite  du 
systeme  AIMS  pour  les  FC  durant  les  operations  de  recherche  et  sauvetage  (SAR).  Done,  en 
utilisant  plusieurs  criteres  d’ evaluation  developpes  dans  des  publications  precedentes,  ce 
memorandum  presente  l’analyse  des  donnees  recueillies  au  SAREX  2007,  ce  qui  augmente 
l’etendue  des  donnees  de  ces  publications. 

Les  conclusions  principales  tirees  de  l’analyse  des  donnees  de  ce  memorandum  sont: 

1.  Precision  de  la  position  d’une  cible  :  Tel  que  prevu,  moins  de  cibles  ont  ete  detectees 
directement  sous  l’avion  puisqu’une  partie  de  cette  zone  est  invisible  aux  techniciens  (techs) 
SAR/observateurs.  Egalement,  toutes  les  cibles  ont  ete  detectees  a  l’interieur  de  0.8  mille 
nautique  (NM),  ce  qui  est  a  l’interieur  du  1  NM  attendu. 

2.  Identification  d’une  cible  :  II  n’a  pas  ete  possible  de  demontrer  entre  la  couleur  et  la  forme 
quel  etait  le  parametre  le  plus  important  pour  detecter  un  objet  dans  son  arriere-plan.  Done,  il 
serait  avantageux  pour  AIMS  d’etre  equipe  d’un  ecran  couleur  a  haute  definition  pour 
maximiser  les  chances  de  detecter  une  cible. 

3.  Nombre  de  retours  en  arriere  :  La  majorite  des  retours  en  arriere  ont  ete  effectues  pour  rien. 
II  serait  preferable  d’ evaluer  ce  critere  durant  une  recherche  ou  il  y  aurait  suffisamment  de 
temps  pour  effectuer  une  bonne  recherche. 

4.  Taux  de  fausses  detections  :  Il  n’a  pas  ete  possible  de  tirer  de  conclusion  a  partir  du  taux  de 
fausses  detections.  Toutefois,  on  suggere  de  comparer  seulement  le  taux  de  fausses  detections 
obtenu  avec  AIMS  a  celui  obtenu  sans  AIMS  pour  le  meme  evenement  SAREX. 

5.  Probability  de  detection  :  Il  n’a  pas  ete  suffisant  d’utiliser  les  donnees  du  SAREX  2007  pour 
tirer  une  conclusion  sur  la  probability  de  detection  (POD).  Par  contre,  en  combinant  les 
resultats  des  SAREX  de  1988  a  1991,  1993  et  2004  a  2007,  on  peut  affirmer  que  la  POD  des 
cibles  passives  est  beaucoup  moindre  que  celle  des  cibles  cooperatives. 

6.  Nombre  de  cibles  :  Les  resultats  obtenus  lors  d’evenements  de  recherche  passes  ont  ete 
compiles  pour  determiner  le  nombre  de  cibles  necessaires  pour  demontrer  avec  95%  de 
confiance  une  amelioration  des  POD.  Etant  donne  qu’on  ne  connait  pas  encore  la  POD  de 
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l’evenement  de  recherche  du  SAREX  auquel  AIMS  participera,  ces  resultats  peuvent 
seulement  etre  utilises  comme  une  indication  du  nombre  de  cibles  a  utiliser.  Neanmoins, 
autant  de  cibles  que  possible  devraient  etre  utilisees  pour  l’evenement  de  recherche  du 
SAREX  auquel  AIMS  participera. 

L’analyste  du  projet  AIMS  DT  (demonstration  de  technologies)  devrait  recueillir  des  donnees  au 
SAREX  2008  et  etre  pret  pour  les  essais  en  vol  avec  AIMS  du  SAREX  2009.  Egalement,  tel  que 
discute  dans  un  memorandum  precedent,  il  est  important  de  mentionner  que  si  1 ’utilisation 
d’AIMS  lors  de  l’evenement  de  recherche  demontre  une  performance  egale  ou  superieure  a  celle 
lorsqu’on  ne  l’utilise  pas,  il  est  alors  possible  de  dire  que  le  systeme  sera  tres  utile  aux  operations 
SAR.  Par  contre,  1’ inverse  ne  serait  pas  necessairement  vrai  puisque  plusieurs  facteurs  presents 
durant  le  SAREX  seraient  absents  lors  d’un  incident  reel  et  pourraient  influencer  negativement  la 
performance  du  systeme  AIMS.  Certains  de  ces  facteurs  sont :  AIMS  ne  volera  pas  avec  le  meme 
type  d’ avion  que  les  autres  equipes  et  il  y  aura  moins  de  techs  SAR/observateurs  a  bord  que 
d’habitude  lors  d’un  evenement  de  recherche  du  SAREX.  Aussi,  apres  les  essais  avec  AIMS  au 
SAREX,  une  seule  serie  de  donnees  sera  recueillie  pour  comparaison  et  aucune  recherche  n’aura 
lieu  la  nuit  et  dans  des  conditions  meteorologiques  degradees,  conditions  pour  lesquelles  le 
systeme  a  ete  specialement  congu  afin  d’ameliorer  l’efficacite  de  la  recherche.  Finalement, 
habituellement  lors  de  l’evenement  de  recherche  du  SAREX,  il  y  a  plus  de  cibles  cooperatives 
que  de  cibles  passives,  alors  que  le  systeme  AIMS  possede  des  capteurs  qui  pourraient  aider  a 
trouver  des  cibles  passives. 


VI 
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1  Introduction 


The  Advanced  Integrated  Multi-sensing  Surveillance  (AIMS)  system  technology  demonstration 
program  (TDP)  was  designed  to  improve  the  surveillance  and  reconnaissance  capabilities  of  the 
Canadian  Forces  (CF),  particularly  at  night  and  in  degraded  weather  conditions.  These  enhanced 
capabilities  for  search  and  rescue  is  possible  due  to  its  integrated  multi-sensors,  such  as  an  active 
imaging  system,  a  thermal  imager,  an  advanced  operator-machine  interface,  an  auto-tracker  and  a 
laser  rangefinder  [1]. 

Since  AIMS  intent  is  to  participate  in  the  Search  and  Rescue  Exercise  (SAREX)  2009,  it  was 
considered  useful  for  the  operational  research  (OR)  analyst  attached  to  the  AIMS  project  to  attend 
previous  SAREX  exercises  to  collect  data  from  experienced  crews  flying  on  different  platforms. 
The  analysis  of  these  data  will  help  evaluate  the  utility  for  the  CF  to  use  the  AIMS  system  for 
Search  and  Rescue  (SAR)  operations.  Therefore,  it  was  essential  to  develop  a  set  of  evaluation 
criteria  (see  Annex  A  for  the  description)  as  reported  in  previous  publications  to  analyze  the  data 
that  were  gathered  during  these  SAREX  exercises  (see  publications  [2] [3] [4] [5]).  Thus,  the  aim  of 
this  memorandum  is  to  improve  the  previously  published  data  collected  during  the  past  SAREX 
events. 

The  first  part  of  this  memorandum  presents  a  short  description  of  the  SAREX  2007  search  event. 
Then,  a  summary  of  the  SAREX  2007  data  analysis  is  presented.  The  data  were  analyzed  using 
the  following  evaluation  criteria:  target  location  accuracy,  target  identification,  number  of  call- 
arounds,  false  detection  rate,  probability  of  detection  and  number  of  targets.  Finally,  a  discussion 
is  presented  and  recommendations  are  provided. 


DRDC  Valcartier  TM  2008-329 


1 


This  page  intentionally  left  blank. 


2 


DRDC  Valcartier  TM  2008-329 


2  SAREX  2007  event 


This  chapter  presents  the  SAREX  2007  search  event  organised  by  444  Squadron  (SQN)  (Goose 
Bay)  that  took  place  in  Goose  Bay  (Labrador)  from  October  2  to  6,  2007  [6].  Also,  the  data 
collected  at  this  SAREX  are  analyzed  and  the  results  are  provided. 

2.1  SAREX  2007  search  event 

Table  1  lists  the  teams  who  took  part  in  the  SAREX  2007  and  provides  the  name  of  squadron  and 
aircraft  used  by  each  team.  Unfortunately,  during  this  yearly  event,  SAREX  2  had  to  go  back  to 
its  base  and  therefore,  this  team  did  not  participate  in  the  search  event. 


Table  1:  SAREX 2007  search  event  teams 


TEAM 

SQUADRON 

HOME  BASE 

AIRCRAFT 

SAREX  1 

435 

Winnipeg 

CC-130  (Hercules) 

SAREX  2 

103 

Gander 

CH-149  (Cormorant) 

SAREX  3 

417-439-444 

Goose  Bay  - 
Bagotville  -  Cold 
Lake 

CH-146  (Griffon) 

SAREX4 

442 

Comox 

CC-130  (Hercules) 

SAREX  5 

424 

Trenton 

CC-130  (Hercules) 

SAREX  6 

413 

Greewood 

CC-130  (Hercules) 

For  the  search  event,  before  the  crews  took  off  for  their  search,  each  team  received  a  Joint  Rescue 
Coordination  Center  (JRCC)  tasking,  as  provided  in  Annex  B,  with  the  mission  to  find  specific 
targets  in  a  defined  area  of  search,  as  shown  in  Figure  1.  On  this  search  map,  the  corners  of  the 
search  area  are  indicated  with  red  flags  and  the  position  of  the  targets  is  indicated  with  yellow 
flags.  Notice  that  all  targets  were  previously  ground-truthed1  and  that  all  teams  were  doing  visual 
detection  only,  which  implies  that  no  external  means  (like  sensors)  other  than  the  eyes  were  used 
to  spot  targets. 


1  In  this  context,  a  target  ground-truthed  is  a  target  placed  on  the  ground  with  a  known  GPS  position. 
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The  description  of  the  targets  is  presented  in  Table  2  and  the  corresponding  pictures  are  provided 
in  Annex  C  (targets  1  to  11).  In  a  previous  memorandum  [5],  it  was  explained  that  a  set  of 
permanent  targets  would  be  implemented  for  each  future  SAREX.  These  targets  correspond  to 
targets  1,  4,  5,  6  and  7  in  Table  2  . 
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Table  2:  Description  of  the  targets  (SAREX  2007)  [7] 


TARGET 

# 

LAT  (N)  /  LONG  (W) 

(DD,MM.M) 

DESCRIPTION 

COOPERATIVE 

TARGET23 

1 

(53°,  3.030’), (-60°,  25.450’) 

Full  Aircraft  (C-JETA) 

YES 

2 

(53°,  8.460’), (-60°,  44.650’) 

Red  Letter  "L" 

NO 

3 

(53°,  2.050’), (-60°,  42.930’) 

Orange  parachute 

YES 

4 

(53°,  7.550’), (-60°,  23.330’) 

White,  yellow  and 
orange  floating  debris 

NO 

5 

(53°,  8.680’), (-60°,  29.270’) 

Floater  (dummy  with 
blue  swim  suit) 

NO 

6 

(53°,  4. 170’), (-60°,  19.230’) 

Parts  of  plane 

NO 

7 

(53°,  6.360’), (-60°,  15.930’) 

Boating  stuff 

YES 

8 

(53°,  1.740’), (-60°,  20.360’) 

Orange  parachute 

YES 

9 

(53°,  2. 180’), (-60°,  19.150’) 

Yellow  parachute 

YES 

10 

(53°,  0.950’), (-60°,  26.230’) 

Yellow  arrow 

YES 

11 

(53°,  7.374’), (-60°,  17.721’) 

White  Lancaster  wreck 

YES 

12 

(53°,  6. 100’), (-60°,  40.560’) 

B36  wreck 

YES 

To  determine  if  a  target  is  cooperative  or  passive  is  sometimes  biased  because  the  decision  can  be 
influenced  by  the  analyst’s  subjectivity.  This  decision  is  usually  based  on  the  type  of  target,  the 
facility  to  see  the  target  from  an  airborne  platform  and  its  contrast  with  the  background 
environment. 

2.2  Target  location  accuracy 

As  done  in  [4] [5],  a  comparison  between  the  sighting  and  the  actual  GPS  position  is  made  to 
evaluate  the  target  location  accuracy.  Figure  2  presents  the  data  collected  at  the  SAREX  2007. 


2  A  cooperative  target  is  a  target  placed  by  survivors  in  an  effort  to  be  seen;  it  could  also  be  a  target  easily 
detected  from  the  air  [5]. 

3  Passive  target  can  be  defined  as  a  target  that  results  directly  from  a  SAR  incident  [5]. 


DRDC  Valcartier  TM  2008-329 


5 


o> 


I  SAREX  2007 


1 


&  &  &  &  .\8>  *S>  _tS>  .t£>  -& 


or  ^  k  \xv 


\"  or  or  ^  «r  *r  a  *r  °>v  K~  v  K „  K 

55*  55*  55*  55*  55*  55  *  55  *  55  *  55  *  55*  \*  N*  N*  \ 

^  <$  <$  &  <§>  &  Ns>  n>  nt  fp  Nr 

Q>*  Q>*  O*  Q>*  Q>*  Q>*  Or  <5*  Q>*  Q>*  N 


& 


\ 


Distance  between  the  aircraft  and  the  target  detected  (NM) 

Figure  2:  Number  of  targets  detected  within  1.5  NM  of  their  exact  position  -  SAREX  2007 


In  Figure  2,  the  data  of  the  first  column  corresponds  to  the  number  of  targets  detected  in  the  area 
directly  beneath  the  aircraft  and  the  subsequent  intervals  contains  the  number  of  targets  detected 
at  a  certain  distance  from  the  aircraft  (here  the  aircraft  is  located  at  0  NM).  It  was  expected  that 
the  majority  of  the  targets  would  be  found  inside  1  NM  of  sight  and  this  is  observed  in  Figure  2. 

Unfortunately,  the  number  of  targets  detected  at  SAREX  2007  was  not  very  high.  This  could  be 
explained  by  several  factors  like  the  bad  weather  during  the  week  and  the  possibility  that  too 
many  targets  where  present  in  the  search  area  for  the  time  given  to  perform  the  search.  In  fact, 
even  if  one  of  the  target  was  large  (a  B36  wreck),  surprisingly,  most  of  the  teams  did  not  see  it. 
Also,  significant  time  was  lost  by  the  first  teams  to  identify  white  "Y"  poles  that  were  not  part  of 
the  scenario.  All  the  subsequent  teams  were  then  advised  not  to  loose  time  to  give  position  and 
description  of  these  poles.  Finally,  some  of  the  targets,  like  the  floater,  even  if  they  were 
representing  a  real  SAR  scenario,  were  probably  too  small  to  be  detected. 

To  increase  the  sample  size  on  location  accuracy,  Figure  3  shows  the  compilation  of  SAREX 
2006  and  2007  data. 
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Distance  between  the  aircraft  and  the  target  detected  (NM) 


Figure  3:  Number  of  targets  detected  within  1.5  NM  of  their  exact  position  - 

SAREX 2006  and  2007 


In  Figure  3,  one  can  see  that  fewer  targets  were  detected  in  the  area  directly  under  the  aircraft. 
This  was  expected  since  the  search  area  directly  beneath  the  aircraft  is  out  of  reach  for  SAR 
technicians  (techs)/spotters.  Also,  the  majority  of  the  targets  were  detected  within  1  NM  of  their 
exact  position. 

2.3  Target  identification 

As  suggested  in  a  previous  document  [5],  the  ability  to  see  colours  and  to  identify  shapes  was  also 
evaluated  using  SAREX  2007  data4.  Table  3  was  used  to  evaluate  the  accuracy  of  the  description 
of  the  targets  given  by  the  teams.  Five  points  were  given  for  each  object  detected  and  another  five 
points  if  at  least  one  of  the  colours  was  identified.  No  points  were  granted  for  a  wrong 
identification  or  bad  colour.  White  and  silver  were  assumed  to  be  the  same  colour  and  orange, 
yellow,  red  and  pink  were  also  considered  to  be  the  same  colour. 
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As  suggested  in  [5],  no  distinction  is  made  between  the  terms  "identification"  and  "recognition". 
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Table  3:  Chart  of  points 


TARGET  # 

OBJECT  (5pts) 

COLOUR  (5pts) 

POINTS 

1 

Full  aircraft 

White  -  black 

10 

2 

L 

Red 

10 

3 

Parachute 

Orange 

10 

4 

Debris  field  (tarp) 

Orange 

10 

Life  preserver 

Orange 

10 

Ice  box 

White 

10 

5 

Dummy 

Blue  -  beige 

10 

6 

Parts  of  plane 

White  -  black 

10 

7 

Floating  debris:  2  paddles 

Beige 

10 

Square  panel 

White  -  blue  -  red 

10 

Lifejacket  (tarp) 

Orange  -  yellow 

10 

8 

Parachute 

White  -  yellow  -  orange 

10 

9 

Parachute 

Yellow 

10 

10 

Arrow 

Yellow 

10 

11 

Lancaster  (Wing) 

White  -  black 

10 

Wheel  (or  debris) 

White  -  black 

10 

12 

B36 

Silver  -  white  -  black 

5 

TOTAL 

165 

Table  4  shows  the  results  of  the  evaluation  of  the  accuracy  of  the  description  of  the  targets  given 
by  the  teams. 
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Table  4:  Evaluation  of  the  accuracy  of  the  description  of  the  targets  given  by  the  teams  at  the 

SAREX 2007  search  event 


TARGET 

# 

SAREX 1 

SAREX  3 

SAREX  4 

SAREX  5 

SAREX  6 

SHAPE/ 

COLOUR 

SHAPE/ 

COLOUR 

SHAPE/ 

COLOUR 

SHAPE/ 

COLOUR 

SHAPE/ 

COLOUR 

1 

5 

0 

- 

- 

5 

- 

- 

2 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

3 

- 

- 

- 

- 

- 

- 

- 

- 

0 

5 

4 

5 

- 

- 

- 

- 

5 

5 

- 

- 

5 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

6 

- 

- 

- 

- 

- 

- 

- 

- 

5 

5 

7 

- 

- 

- 

- 

- 

- 

- 

- 

8 

- 

- 

- 

- 

- 

- 

- 

- 

0 

5 

9 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

10 

5 

5 

- 

- 

5 

5 

5 

5 

- 

- 

11 

- 

- 

10 

5 

5 

0 

- 

- 

- 

- 

12 

- 

- 

- 

- 

- 

- 

- 

- 

5 

5 

TOTAL: 

#  PTS 

%  (max=) 

10 

10 

10 

5 

15 

5 

15 

15 

10 

20 

6.1 

6.1 

6.1 

3 

9.1 

3 

9.1 

9.1 

6.1 

12.2 

Since  there  were  not  many  data  available  to  perform  this  analysis,  we  should  be  careful  in 
interpreting  the  results.  An  example  of  how  to  complete  this  table  is  explained  below. 

Let  us  take  the  results  of  SAREX  1  indicated  in  Table  4.  Target  1  was  detected  and  since  in  the 
sheet  results  it  was  specified  that  it  was  a  fuselage,  five  points  were  given  for  the  recognition  of 
the  fuselage  shape,  but  no  point  was  awarded  for  the  colour.  On  the  other  hand,  target  10  was 
detected  and  identified  by  a  red  arrow  pointing  east;  therefore,  five  points  could  be  given  for  the 
identification  of  the  shape  and  five  points  for  the  good  colour.  Once  this  procedure  was  applied 
for  all  the  targets  detected  by  each  team,  the  points  were  summed  in  each  category  (colour  and 
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shape)  and  were  divided  by  the  maximum  total  of  points  possible  (165).  The  percentage  obtained 
(in  this  case  SAREX  1  scored  6.1%  for  shape  and  6.1%  for  colour)  was  then  reported  in  Figure  4. 

Figure  4  shows  the  percentage  obtained  by  each  team  for  a  good  identification  of  colour  versus 
the  percentage  obtained  for  a  good  identification  of  shape. 
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Figure  4:  Percentage  obtained  for  good  identification  of  shape  versus  percentage  obtained  for 
identification  of  at  least  one  good  colour  -  SAREX  2007 
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In  Figure  4,  the  low  percentage  obtained  in  each  category  is  simply  reflecting  the  fact  that  only  a 
few  targets  were  detected  at  this  search  event.  Also,  the  results  above  the  doted  line  (equivalence 
line)  would  mean  that  the  colour  parameter  helped  more  than  the  shape  parameter  to  identify  a 
target  and  that  would  mean  the  opposite  for  the  results  below  this  line.  In  this  case,  due  to  the 
small  number  of  data  collected  at  the  SAREX  2007,  it  is  not  possible  to  conclude  which  one  of 
the  shape  or  colour  is  the  most  important  discrimination  parameter. 

It  is  important  to  mention  that  several  factors  may  have  influenced  the  results.  Firstly,  when  SAR 
techs/spotters  provide  information  about  a  potential  target,  the  colour  or  the  shape  is  not 
necessarily  noted  on  the  sheet  results.  From  what  have  been  seen,  SAR  techs/spotters  will  only 
describe  the  aspect  that  will  help  others  to  identify  the  object  as  soon  as  possible.  If,  for  example, 
a  white  fuselage  is  crashed  in  a  snow  background,  the  white  colour  will  probably  not  be  the 
turning  point.  However,  in  the  case  of  an  orange  parachute  found  on  the  ground,  the  orange 
colour  will  be  more  attractive  in  the  middle  of  the  woods  than  the  type  of  object.  Secondly,  the 
initial  choice  of  targets  could  also  influence  the  results  in  the  sense  that  if  we  have  four  targets  to 
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find  and  three  of  them  are  orange  parachutes  and  if  only  one  is  a  white  fuselage,  the  colour 
parameter  in  this  search  will  probably  appear  to  be  more  important. 

To  increase  the  number  of  data,  a  combination  of  SAREX  2006  and  SAREX  2007  analysis  of 
shape  and  colour  is  presented  in  Figure  5. 
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Figure  5:  Percentage  obtained  for  good  identification  of  shape  versus  percentage  obtained  for 
identification  of  at  least  one  good  colour  -  SAREX  2006  and  2007 
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In  Figure  5,  the  colour  parameter  seems  to  be  more  important  than  the  shape  for  the  identification 
of  a  target,  but  again,  due  to  the  small  number  of  data,  it  is  not  possible  to  conclude  on  which  one 
of  the  shape  or  colour  is  the  most  important  discrimination  parameter.  Therefore,  since  we  cannot 
conclude  for  the  moment  with  these  results,  it  could  be  suitable  for  AIMS  to  be  equipped  of  a 
high  definition  colour  screen  to  maximize  the  chances  of  detecting  a  target. 

2.4  Number  of  call-arounds 

During  SAREX  2007,  search  paths  were  recorded  and  provided  by  Messrs.  Taylor  and  Saunders 
[8].  Using  the  free  trial  version  of  OziExplorer  [9],  a  visual  map  was  generated  for  each  aerial 
track  flown.  An  example  of  GPS  track  of  SAREX  1  is  shown  in  Figure  6.  To  lighten  this  section, 
the  search  paths  recorded  for  SAREX  3-4-5  and  6  are  presented  in  Annex  D. 
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Figure  6:  SAREX 1  GPS  track  [8] 


In  Figure  6,  one  can  observe  that  a  call-around  is  made  each  time  the  aircraft  is  doing  a  circle 
over  a  particular  area.  It  is  necessary  to  perform  a  call-around  when  SAR  techs/spotters  think  they 
have  seen  something  but  cannot  confirm  what  it  is.  In  that  case,  the  aircraft  turns  around  and 
performs  circles  around  the  potential  target  until  it  is  possible  for  the  SAR  techs/spotters  to 
confirm  if  it  is  or  not  a  target. 

The  number  of  call-arounds  compiled  during  SAREX  2007  is  provided  in  Table  5. 
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Table  5:  Number  of  call-arounds 


TEAM 

ON  DETECTED 
TARGETS 

ON  MISSED 
TARGETS 

FOR 

NOTHING 

TOTAL 

SAREX 1 

0 

0 

oo 

O© 

SAREX 3 

1 

0 

11 

12 

SAREX  4 

1 

0 

6 

7 

SAREX 5 

1 

0 

3 

4 

SAREX  6 

4 

0 

4 

TOTAL 

7 

0 

32 

39 

In  a  previous  publication  [5],  2/3  of  call-arounds  were  made  for  nothing,  in  SAREX  2008,  4/5  of 
call-arounds  were  made  for  nothing.  Different  factors  could  be  given: 

•  SAREX  3  did  not  have  the  time  to  fly  the  entire  area. 

•  SAREX  5  experienced  very  bad  weather,  both  times  they  flew. 

•  SAREX  6  started  their  path  on  the  perimeter  of  the  search  instead  of  being  1NM  inside 
the  parameters  of  search.  Consequently,  the  last  1  NM  was  not  monitored. 

Also,  as  previously  mentioned  in  [5],  it  would  be  preferable  to  perform  the  evaluation  of  this 
criterion  during  a  search  where  there  would  be  sufficient  time  to  perform  a  good  search  so  that 
call-arounds  can  be  made  more  frequently,  if  needed. 

2.5  False  detection  rate 

Table  6  summarizes  the  false  detection  rates  obtained  at  the  SAREX  2007  search  event. 
Considering  that  the  SAR  techs/spotters  could  cover  approximately  1  NM  on  each  side  of  the 
aircraft,  the  area  to  search  was  divided  into  five  corridors.  To  be  able  to  determine  the  dimensions 
of  the  area  of  search  in  NM,  first,  the  coordinates  were  obtained  in  decimal  using  the  converter 
given  in  [10].  Then,  using  another  method  provided  in  [11],  it  was  possible  to  calculate  the 
distance.  The  area  of  search  was: 


\S.072[NM]x\0.0l2[NM]  =  m.93l[NM2]  (1) 
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Average  of  false  targets  reported/NMA2 


Table  6:  SAREX 2007 false  detections  rates 


TEAM 

#FALSE  TARGETS 

REPORTED 

FALSE  DETECTION  RATES 

(#FALSE  TARGETS  /  NM2) 

SAREX 1 

15 

0.0829 

SAREX 3 

4 

0.0221 

SAREX  4 

19 

0.1050 

SAREX 5 

6 

0.0332 

SAREX  6 

14 

0.0774 

AVERAGE 

11.6 

0.0641 

0.08 

0.07 

0.06 

0.05 

0.04 

0.03 

0.02 

0.01 

0 


In  Figure  7,  one  can  observe  that  the  false  detection  rates  do  not  follow  any  particular  trend. 


■  False  detection  rates 


1 988  -  1 989  -  1 990  -  1 991  -  1 993  -  2005  -  2006  -  2007  - 

Comox  Trenton  Edmonton  Comox  Trenton  PEI  North  Bay  Goose  Bay 


Year 

Figure  7:  False  detection  rates 
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1 988  -  1 989  -  1 990  -  1 991  -  1 993  -  2005  -  PEI  2006  -  2007  - 

Comox  Trenton  Edmonton  Comox  Trenton  North  Bay  Goose  Bay 

Year 


Figure  8:  Comparison  between  real  target  detected  and  false  target  reported 


Figure  8  does  not  show  any  relationship  between  the  average  number  of  targets  detected  and  the 
average  number  of  false  targets  reported.  As  discussed  in  [5],  the  false  detection  rate  obtained 
using  AIMS  should  only  be  compared  to  that  obtained  without  AIMS  for  the  same  SAREX  event. 

2.6  Probability  of  detection 

This  section  provides  results  on  the  probability  of  detection  (POD)  and  presents  many  factors  that 
could  influence  those  POD. 

Table  7  summarizes  the  POD  of  different  targets  at  SAREX  2007. 
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Table  7:  Estimated  POD  of  SAREX 2007 


TARGET 

ATTEMPTS 

DETECTIONS 

FRACTION 

DETECTED 

1 

Full  Aircraft,  white  (C-JETA) 

5 

3 

0.6 

2 

Red  L 

5 

0 

0 

3 

Para 

5 

1 

0.2 

4 

Debris 

5 

2 

0.4 

5 

Floaters 

5 

0 

0 

6 

Parts  of  plane 

5 

1 

0.2 

7 

Boating  stuff 

5 

1 

0.2 

8 

Chute 

5 

1 

0.2 

9 

Chute  2 

5 

0 

0 

10 

Arrow 

5 

3 

0.6 

11 

Lancaster 

5 

2 

0.4 

12 

B36 

5 

1 

0.2 

All  cooperative  targets 

40 

12 

0.30 

All  passive  targets 

20 

3 

0.15 

In  a  previous  publication  [5],  it  was  suggested  to  introduce  a  new  set  of  targets  of  reference 
during  the  SAREX  search  event  until  the  flight  trials  with  AIMS.  The  description  of  these  targets 
is  provided  in  Annex  E.  During  SAREX  2007,  these  targets  corresponded  to  targets  1,  4,  5,  6  and 
7  in  Table  2.  Some  modifications  were  done  to  the  previous  list  due  to  material  availability  and 
the  feasibility  of  placing  the  targets  on  site.  Amongst  other  things,  the  panel  with  letters  was 


16 


DRDC  Valcartier  TM  2008-329 


replaced  by  a  red  L  and  the  floating  debris  were  combined  with  the  panel  with  reflective  strips 
submerged. 

Previous  analysis  of  SAREX  data  [3] [4] [5] [12] [13] [14]  demonstrated  that  the  POD  for  passive 
targets  is  significantly  less  than  that  for  cooperative  targets.  In  Figure  9,  one  can  observe  the  POD 
for  SAREX  2007  with  confidence  intervals  of  90%,  95%  and  99%  and,  in  Figure  10,  the 
estimated  POD  for  cooperative  and  passive  targets  with  confidence  intervals  of  90%,  95%  and 
99%  for  SAREX  1988  to  1991,  1993  and  2004  to  2007. 


cooperative  target 
passive  target 

0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1 

Probability  of  detection 

■  99%  □  95%  □  90%  confidence  intervals 

Figure  9:  Estimated  POD  for  cooperative  and  passive  targets  for  SAREX  2007 


cooperative  target 

i  r 

i  r 

■ 

□0 

passive  target 

m  f 

n 

0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9  1 

Probability  of  detection 


■  99%  H95%  □  90%  confidence  intervals 


Figure  10:  Estimated  POD  for  cooperative  and  passive  targets  for  past 
SAREX  (1988  to  1991 ,  1993,  2004  to  2007) 


In  Figure  9,  only  the  data  obtained  in  SAREX  2007  are  presented.  As  one  can  see,  the  confidence 
intervals  intersect.  However,  if  a  sufficient  number  of  data  is  combined,  one  can  observe  in 
Figure  10  that  we  support  the  following  conclusion:  passive  targets  are  significantly  harder  to  find 
than  cooperative  targets. 
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In  a  previous  memorandum  [5],  a  summary  of  the  factors  that  could  influence  the  POD  were 
brought  together: 

•  Weather  and  time  of  day  (clouds,  lighting,  fog,  precipitation);  the  weather  might  be  different 
each  year  and  might  be  different  for  different  teams. 

•  The  event  location  where  the  searches  take  place  is  different  each  year,  therefore,  the 
background  contrast  and  the  environment  also  change. 

•  Search  and  aircraft  parameters  (size  of  the  area  to  search,  type  of  aircraft,  airspeed,  altitude, 
track  spacing,  sweep  width,  time  flying). 

•  Condition  of  the  crew  (crew  fatigue,  crew  motivation,  experience  of  crew  members  is  not 
constant  within  a  team  and  between  the  teams). 

•  Number  of  SAR  techs/spotters  might  change  between  aircraft;  there  are  usually  more  SAR 
techs/spotters  during  SAREX  searches  than  there  are  during  real  SAR  incidents  and  SAR 
techs/spotters  search  for  the  entire  hour  during  the  search  rather  than  alternate  every  20-30 
minutes  as  is  standard  practice. 

•  Target  characteristics:  each  year  the  number  and  type  of  targets  change.  Thus,  it  is  difficult 
to  establish  a  solid  baseline  for  comparison  purposes. 

•  Subjectivity  of  the  scientist  to  class  a  target  as  passive  or  cooperative  when  calculating  the 
POD. 

•  Ability  of  the  sensor  to  sweep  the  entire  area  as  the  aircraft  flies  over. 

•  Familiarity  with  the  environment. 

2.7  Number  of  targets 

A  method  was  developed  to  determine  the  number  of  targets  that  are  necessary  to  demonstrate 
with  a  certain  level  of  statistical  confidence  an  increase  in  POD  [3]  [4].  A  FORTRAN  program 
was  used  to  determine  the  confidence  rate  of  SAREX  2007  data  and  is  provided  in  Annex  F.  This 
program  is  equivalent  to  the  Matlab  program  provided  by  [15]  and  that  was  presented  in  [5].  The 
results  obtained  using  SAREX  2007  are  given  in  Figure  1 1  and  for  comparison  purposes,  the  data 
compiled  using  past  SAREX  are  also  presented.  The  results  from  past  SAREX  were  extracted 
from  [5]  and  exclude  SAREX  2007  results. 
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Figure  11:  Detection  rate  required  to  demonstrate  improvements  with  95%  confidence  using 
SAREX  2007  and  using  past  SAREX  search  event  data 


As  explained  in  [3],  Figure  1 1  shows  "the  proportion  of  targets  that  must  be  detected  by  AIMS  in 
order  to  demonstrate  improvements  in  performance  with  95%  confidence  as  a  function  of  the 
number  of  targets  used". 

SAREX  2007  POD  for  passive  targets  was  0.15  and  that  of  past  SAREX  (excluding  SAREX 
2007)  was  0.21.  The  curves  of  Figure  1 1  were  built  using  the  POD  and  the  Bayesian/Monte-Carlo 
approach  [3]  [4].  An  example  of  how  to  read  this  figure  is  explained  below. 

If  AIMS  is  expected,  for  example,  to  have  a  POD  for  passive  targets  of  65%,  the  SAREX  search 
event  would  be  appropriate  to  show  its  superiority  if  at  least  three  or  more  passive  targets  were 
used  (using  SAREX  2007  data  in  this  example).  Note  that  this  figure  is  only  an  indication  of  the 
detection  rate  versus  the  number  of  targets  since  we  do  not  know  yet  the  POD  of  the  search  event 
in  which  AIMS  will  be  participating.  Of  course,  for  this  search  event  as  many  targets  as  possible 
should  be  employed. 
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3  Conclusion 


Since  AIMS  intent  is  to  participate  in  the  Search  and  Rescue  Exercise  (SAREX)  2009,  it  was 
considered  useful  for  the  operational  research  (OR)  analyst  attached  to  the  AIMS  project  to  attend 
previous  SAREX  exercises  to  collect  data  from  experienced  crews  flying  on  different  platforms. 

The  aim  of  this  memorandum  was  to  analyze  the  data  collected  at  the  SAREX  search  event  2007 
and  to  improve  the  previous  published  database.  The  evaluation  criteria  provided  in  [5]  were  used 
to  analyze  the  data.  The  main  conclusions  drawn  from  this  analysis  are: 

1.  Target  location  accuracy:  As  expected,  few  targets  were  detected  under  the  aircraft  since 
part  of  this  area  is  out  of  reach  for  SAR  technicians  (techs)/spotters.  Also,  all  the  targets  were 
detected  within  0.8  nautical  miles  (NM),  which  is  within  the  1  NM  expected. 

2.  Target  identification:  It  was  not  possible  to  demonstrate  which  one,  between  colour  and 
shape,  was  the  most  useful  parameter  to  detect  an  object  from  its  background.  Therefore,  it 
could  be  suitable  for  AIMS  to  be  equipped  with  a  high  definition  colour  screen  to  maximize 
the  chances  of  detecting  a  target. 

3.  Number  of  call-arounds:  The  majority  of  call-arounds  were  made  for  nothing.  It  would  be 
preferable  to  evaluate  this  criterion  during  a  search  when  there  would  be  sufficient  time  to 
perform  a  good  search. 

4.  False  detection  rate:  It  was  not  possible  to  draw  any  conclusion  from  the  false  detection  rate. 
It  is  suggested  to  compare  only  the  false  detection  rate  obtained  using  AIMS  to  that  obtained 
without  AIMS  for  the  same  SAREX  event. 

5.  Probability  of  detection:  It  was  not  sufficient  to  use  only  the  data  obtained  from  SAREX 
2007  to  reach  a  conclusion  on  the  probability  of  detection  (POD).  However,  combining  the 
data  from  1988  to  1991,  1993  and  2004  to  2007,  we  can  confirm  that  the  POD  for  passive 
targets  is  significantly  less  than  that  for  cooperative  targets. 

6.  Number  of  targets:  Past  search  events  data  were  compiled  to  determine  the  number  of 
targets  that  were  necessary  to  demonstrate,  with  95%  confidence,  an  improvement  in  the 
POD.  Since  we  do  not  know  yet  the  POD  of  the  SAREX  search  event  in  which  AIMS  will 
participate,  these  results  can  only  be  used  as  an  indication  of  the  number  of  targets  required. 
Nevertheless,  as  many  targets  as  possible  should  be  employed  for  the  SAREX  search  event  in 
which  AIMS  will  participate. 

For  SAREX  2007,  not  many  data  were  collected.  This  is  mainly  due  to  the  fact  that  1)  the  weather 
was  not  good  during  almost  all  the  week  and  2)  the  area  of  search  was  too  large  for  the  time 
given. 

It  was  possible  to  include  the  targets  provided  in  the  reference  set.  It  is  strongly  recommended  to 
include  these  targets  for  SAREX  2008  and  for  SAREX  2009  when  AIMS  will  be  flying. 
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It  was  also  observed  at  SAREX  2006  (North  Bay)  and  SAREX  2007  (Goose  Bay)  that  the  end  of 
September  and  the  beginning  of  October  might  not  be  the  best  time  for  SAREX  events.  Holding 
the  event  earlier  in  the  season  (for  example  at  the  beginning  of  September)  could  probably 
increase  the  chances  of  good  weather. 
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Annex  A  Measures  of  effectiveness 


Evaluation  criteria  (which  include  MOEs  and  additional  criteria  that  are  not  MOEs)  were 
developed  to  evaluate  the  AIMS  TD  utility  for  the  CF  in  SAR  operations.  The  MOEs  proposed  in 
[5]  are: 

1.  The  alignment  error  between  the  actual  location  of  the  target  and  the  one  given  by  the 
tracking  device. 

2.  The  maximum  time  that  the  sensor  is  able  to  stay  on  the  target. 

3.  The  average  time  it  takes  for  the  sensor  to  recover  the  target  when  loosing  it. 

4.  The  time  from  fly-over  to  detection. 

5.  The  time  necessary  to  plan  an  optimal  search. 

6.  The  number  of  hours  of  experience  as  a  SAR  tech/spotter  (spotting  with  naked  eyes)  and  as 
an  operator  using  the  sensor  suite. 

7.  The  maximum  time  that  the  sensor  suite  can  be  used  without  significant  operator  fatigue. 

8.  The  maximum  flight  velocity  for  the  operator  to  be  able  to  analyze  effectively  the  data  in  real 
time. 

9.  The  precision  of  the  geo-location  system. 

10.  The  average  time  required  between  the  detection  and  the  identification/recognition  of  a  target. 

1 1 .  The  number  of  call-arounds. 

12.  The  average  number  of  false  positives  per  unit  of  area. 

13.  The  probability  of  detection. 

14.  The  maximum  detection  range  of  the  sensor  suite. 

And  the  other  criteria  proposed  in  [5]  are: 

15.  The  presence  of  a  capability  to  store  captured  images. 

16.  The  ability  of  the  sensor  suite  to  see  colours  and  identify  shapes. 

17.  The  ability  or  effectiveness  of  the  sensor  suite  to  remain  effective  during  different  visibility 
and  illumination  conditions. 

18.  The  presence  of  a  reasonable  image  transmission  capability. 
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Annex  B  JRCC  tasking 


The  SAREX  2007  search  event  is  detailed  in  B.l  and  B.2  [7], 


B.1  Search  event  scenario 

Overdue  Cessna  172,  G-CFOX  departed  Goose  Bay  on  a  VFR  flight  plan  and  headed  south  of  the 
airport  to  conduct  some  aerial  photography.  Two  persons  on  board,  both  male,  mid  20 ’s  with  no 
health  problems.  Last  transmission  to  Goose  Bay  tower  indicated  they  were  experiencing  engine 
problems  and  could  not  maintain  altitude. 

B.2  JRCC  tasking 

Complete  a  visual  search  in  area  bounded  by  the  following  coordinates;  5300N  06045 W,  53 10N 
06045W,  53 10N  06015W,  5300N  06015W.  You  have  one  hour  to  complete  a  search  of  the  area. 
Mark  suspicious  sightings  on  map  and  put  lat/long  and  description  on  form  below. 


Target  # 

LAT/LONG 

DESCRIPTION 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 
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Annex  C  Targets  pictures 


Annex  C  contains  a  series  of  pictures  of  the  targets  that  were  taken  at  the  SAREX  2007  by 
members  of  the  SAREX  committee  [7]  and  by  the  author  of  this  memorandum.  Figure  B-l  and  B- 
2  represent  a  crashed  aircraft.  For  comparison  purposes,  two  persons  are  shown  in  Figure  B-l. 


Figure  B-l:  Target  1  -  Full  aircraft  -  View  A 
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Figure  B-2:  Target  1  -  Full  aircraft  -  View  B 
Figure  B-3  shows  a  red  L  hidden  in  a  very  small  open  area. 


Figure  B-3:  Target  2  -  Red  L 
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Figure  B-4:  Target  3  -  Parachute  1 


Figure  B-5:  Target  4  -  Debris  field 
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Figure  B-7:  Target  5  -  Floater  -  View  B 
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Figure  B-8  shows  parts  of  a  burned  fuselage. 


Figure  B-8:  Target  6  -  Debris  field 


Figure  B-9  shows  floating  debris  (paddle,  lifejacket,  square  plate  with  circle). 


Figure  B-9:  Target  7  -  Floating  debris 
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Figure  B-ll:  Target  9  -  Parachute  3 


34 


DRDC  Valcartier  TM  2008-329 


Figure  B-12:  Target  10  -  Yellow  arrow 
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Figure  B-13:  Target  11  -  Lancaster  wreck  -  View  A 


Figure  B-14:  Target  11  -  Lancaster  wreck  -  View  B 
No  picture  was  provided  for  target  12  (B36  wreck). 
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Annex  D  Examples  of  GPS  tracks 


The  map  of  the  search  was  provided  by  [7]  and  SAREX  2007  GPS  tracks  were  provided  by  [8] 
and  are  shown  below. 

Figure  D-l  shows  the  GPS  tracks  on  the  search  map  for  the  SAREX  3  team. 


Figure  D-l:  SAREX 3  GPS  track 


Figure  D-2  shows  the  GPS  tracks  on  the  search  map  for  the  SAREX  4  team. 


Figure  D-2:  SAREX  4  GPS  track 
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Figure  D-3  shows  the  GPS  tracks  on  the  search  map  for  the  SAREX  5  team  (first  flight  of  two). 


Figure  D-3:  SAREX  5  GPS  track  -  Flight  1 

Figure  D-4  shows  the  GPS  tracks  on  the  search  map  for  the  SAREX  5  team  (flight  two). 


Figure  D-4:  SAREX  5  GPS  track  -  Flight  2 
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Figure  D-5  shows  the  GPS  tracks  on  the  search  map  for  the  SAREX  6  team. 
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Annex  E  Description  of  reference  targets 


The  following  targets  were  chosen  as  a  set  of  reference  for  future  SAREX  [5]: 

•  TARGET  1:  Panel  with  letters  on  it  (for  example:  SOS)  with  a  reasonable  height. 

•  TARGET  2:  Intact  fuselage  with  identification  through  trees  (target  smaller  than  a  real 
fuselage  but  will  have  the  same  coating  as  a  real  one;  target  has  to  be  small  since  it  is  carried 
by  helicopter). 

•  TARGET  3:  Fuselage  ruined/burned  (it  will  be  a  matte  black  frame  with  pipes,  maybe  one 
or  two  small  panels  to  represent  a  seat).  It  happened  in  the  past  that  a  crash  caused  only  a 
flash  of  fire,  leaving  no  trace  of  the  accident  except  a  bundle  of  pipes. 

•  TARGET  4:  Bundle  of  small  targets  (parachute,  dummy,  reflective  tape,  icebox)  dispersed 
in  either  a  swamp  area,  open  area,  or  where  trees  are  cut  (to  simulate  the  action  of  wild 
animals  dispersing  objects). 

•  TARGET  5:  Floating  debris  (for  example,  floating  life  preserver,  reflective  strip  with 
letters,  geometric  shapes). 

•  TARGET  6:  Object  under  the  surface  of  the  water  (panel  with  reflective  strips  submerged 
at  1  foot  or  2  feet  with  4  life  buoys  to  maintain  its  level  of  floatability). 

•  TARGET  7:  Drowned  person  (a  person  in  t-shirt  and  jeans  floating  in  the  water). 
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Annex  F  FORTRAN  program 


The  FORTRAN  program  used  to  determine  the  confidence  rate  that  a  particular  method  is  better 
than  another  one  is  equivalent  to  the  Matlab  program  provided  by  [15]  and  that  was  presented  in 

[51- 

program  test3 
use  dflib 

implicit  none 
integer  nl,  n2,  kl,  k2,  i,  it 
integer  y,  n,  jj,  nsets 
character  b 
real  res,  invres 

real* 8  bayesian 
write(*,*)  'enter  i,  it' 
read(*,*)  i,it 

write(*,*)  'enter  number  of  sets' 
read(*,*)  nsets 
do  jj=l,  nsets 

write(*,*)  'enter  nl,n2,kl,k2' 
read(*,*)  nl,n2,kl,k2 
res  =  bayesian(nl,  kl,  n2,  k2,  i,  it) 
invres  =  (1. -res)*  100 
do  while  (invres  .LT.  95.) 
k2=k2+l 

write(*,*)  'nl,  n2,  kl,  k2- ,nl,n2,kl,k2 
res  =  bayesian(nl,  kl,  n2,  k2,  i,  it) 
invres  =  (1. -res)  *100 
end  do 

write(*,*)  'bayesian-,  k2,  res*100.,  (l.-res)*100. 

C  THE  FUNCTION  RETURNS  THE  CONFIDENCE  RATE  THAT  THE  DETECTION  RATE 
C  OF  METHOD  1  (nl  detections,  kl  independent  trials)  IS  BETTER  THAN 
C  THE  DETECTION  RATE  OF  METHOD  2  (n2  detections,  k2  independent  trials) 
end  do 
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b=getcharQQ() 

end 

real*8  function  bayesian(nl,kl,n2,k2,i,it) 
implicit  none 
integer  nl,  n2,  kl,  k2,  i,  it 
integer  j,  y,  n,  z,  k,  m,  kkl,  kk2,  wl,  w2,  w3 
integer  nlkkl,  n2kk2 
real* 8  ps(2,i) 

real* 8  s,  d,  ss,  c,  ncl,  nc2,  mm,  kn2 

real*  8  p  (it) 

real* 8  pp  (2, it) 

real* 8  nchoosek 

do  j=l,i 

ps(l,j)=0. 

Ps(2,j)=0. 
end  do 
n=nl+n2 
y=kl+k2 
do  j=l,i 

ps(l,j)=j*1.0/i-1.0/(2*i) 

ps(2,j)=(ps(  1  ,j)*  *(y-0.5))*((  1  -ps(  1  ,j))**(n-y-0.5)) 
end  do 
s  —  0 

do  10  m=l,i 

s  =  s  +  ps(2,m) 

10  continue 
do  j=l,i 

ps(2,j)=ps(2,j)*  1.0/s 
end  do 
do  j=2,i 

Ps(2,j)  =  ps(2,j)+ps(2,j-l); 
end  do 

call  random  seed  () 
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call  random_number  (p) 
do  z=l,it 
pp(l,z)=0. 
pp(2,z)=0. 
end  do 

d=k2*  1 ,0/n2-kl  *  1 .0/nl 
do  j=l,it 
k=l 

do  while  (p(j)  .GT.  ps(2,k)) 
k=k+l 
end  do 

pp(l,j)  =  ps(l,k) 
pp(2,j)  =  0 
do  kk2=0,n2 
kn2=kk2*1.0/n2 
mm=nl  *(kn2+d) 
kkl  =  ceiling(mm) 
ss  =  0 

do  while  (kkl  .LE.  nl) 
nc  1  =nchoosek(n  1  ,kkl ) 

ss  =  ss  +  ncl*(pp(l,j)**kkl)*((l-pp(l,j))**(nl-kkl)) 
kkl  =  kkl  +  1 
end  do 

nc2=nchoosek(n2  ,kk2) 

PP(2,j)  =  pp(2,j)  +  ss*nc2*(pp(l,j)**kk2)* 

&((  1  -PP(  1  j  ))*  *  (n2-kk2)) 
end  do 
end  do 

C  DETERMINE  THE  SUM  OF  C 
c  =  0 

do  11  j=l,it 
c=c+pp(2,j) 

1 1  continue 
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C  DIVIDE  THE  SUM  OF  C  BY  THE  NUMBER  OF  ELEMENTS  IT 
bayesian  =  c/it 
return 
end 

C  THIS  FUNCTION  IS  SIMILAR  TO  THE  FUNCTION  NCHOOSEK  IN  MATLAB 
real* 8  function  nchoosek(nl,kkl) 
implicit  none 
integer  nl,  kkl,  j,  nlkkl 
real* 8  wl,  w2,  w3 


C  DETERMINE  nl!/((nl-kkl)!kkl!) 

C  DETERMINE  nl! 
wl=l 

do  13  j=l,nl 
w  1  —  w  1  *  j 

13  continue 

C  DETERMINE  kkl! 
w2=l 

do  14j=l,kkl 
w2=w2*j 

14  continue 

C  DETERMINE  (nl-kkl)! 
nlkkl=nl-kkl 
w3=l 

do  15  j=l,  nlkkl 
w3=w3*j 

15  continue 

C  RETURN  THE  VALUE  OF  nchoosek 
nchoosek  =  wl*1.0/(w2*w3) 
return 
end 
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